Suggestions to keep Beta releases from messing up production servers

[dgaschk]

I think naive users treat unRAID betas like other betas downloaded from the Internet. E.g., a video player beta might work well enough and if it crashes no real harm is done. People don't seem to realize that storage server betas may cause data loss.

[PeterB]

It apparently is only the user with an HPA that subsequently removed it that hit the bug in 5.0bet6. (at least that is Tom's theory right now)

 

If it's his theory on the MBR not recognized in 5.0 beta 6, then the HPA theory holds no water. I never had HPA on any of my drives and have never removed it on any of the drives.

 

Hmm ... well there must be some common factor between those who have experienced the problem with b6!

(Un)fortunately, with computers, there should be none of the "it's the exception which proves the rule"!

 

Were all your drives 'brand new' to you - ex-manufacturer?

Have you ever performed any manipulation on your drives which is out of the ordinary 'install, pre-clear, format, use' cycle?

[BRiT]

Nope, all my drives were purchased brand new and ran through 3 cycles of PRECLEAR. I did have 1 RMA drive, which was run through 5 cycles of PRECLEAR (one of the Seagate 5900rpm).

 

The WD EADs (Sector 63 aligned) were not new neither were the Seagate 5900rpms (Sector 64 aligned). The drives have only ever been used on a Jetway motherboard and the widely used MSI H55 motherboard, have never been connected to anything Gigabyte, the logs never showed HPA or anything odd. Only the Seagates were connected to the IBM BR10i LSI1068E based controller, the others were purely only on MB Sata ports.

 

I never did any manipulation of the drives. The WD's have been running for 19 months or so in unRAID 4.5 betas, then unraid 4.5, then unraid 5.0 beta 1 and 2, then hit up unRAID 5.0 beta 3, then 5.0 beta 4, then beta 5b (skipped beta 5a), then beta 6.

 

The Seagates have been running in unRAID 5.0 beta 4, then beta 5b (skipped beta 5a), then beta 6.

 

I never did any manipulation to the drivers other than to fix the MBR issue that unRAID 5.0 beta 6 caused.

[lionelhutz]

Thanks for bringing it over here.

 

I don't know the exact answer but I just didn't feel that having a beta release "front and center" on the download page was a good idea. Maybe Tom will read the opinions here before deciding what to do.

 

I don't have a test server so I will likely be looking at a switch if a 5.0b release shows very high promise or when a RC is released. Either way, I applaud those who do switch and test the beta releases out.

 

My understanding is that 5.0b2 was mostly to convert the interface and some security changes. Now Tom is really throwing some major re-write changes into the mix and the betas may be very buggy for the next while.

 

Peter

[SSD]

I really think we need to reconsider the meanings of these terms in relation to unRAID

 

alpha test

beta test

release candidate

release version

 

Below would be my recommendation:

 

Implement a closed ALPHA test group.  This group would test out new versions on test arrays only, looking for the more obvious and serious problems.  Since the ALPHA test group would be considerably larger with more diverse arrays than Tom's internal testing pool, it is likely that ALPHA testing would find and allow Tom to address a set of issues prior to beta.  Once the alpha testers have installed the new version, verified the upgrade process worked and their arrays are functional afterwards, run through some of the new functionality, they should report bugs or give it a thumbs up to move to beta.  This criteria would address a good % of early problems, and keep alpha test cycles short.  For minor releases this ALPHA test might be very short.

 

Once ALPHA testing is complete, BETA testing should begin.  BETA testing should be longer.  All normal precautions and disclaimers would apply, but it would be open and available to anyone.  The early adopters would still take their lumps, but not as many as today.  Over time the beta would get more adopters and the "ingenious idiott" testing necessary to really find problems.  Towards the end for BETAs that look to be moving toward release candidates, we should have some test group, similar to what I organized HERE, to run it through its paces - more than just random use.

 

From BETA, there are two options.

 

1 - The version becomes a release candidate

2 - New functionality is added, and a new closed alpha release comes out

 

Once a beta release is solid, it should become a release candidate (RC).  Although similiar to a BETA, an RC release would be adopted by an even wider group of users than a BETA release, and hopefully uncover a few more defects.  RC should quickly be promoted to release if bugs are not found in a defined period (say 7 days or 10 days).

 

Thoughts?

[Rajahal]

That's basically the same thing that I suggested except that I believe alpha releases should be kept in house (LimeTech's servers only), and betas should take the place of what you describe as alphas (closed group testing only).  The RCs would then take the place of what you described as betas (wide-spread testing by the whole community, cue ingenious idiot effect).  As I understand it, that's the path that most software development takes.  Is there a problem with this method, or are we just arguing semantics?

[BRiT]

In either case, I would like the ability to sign up for any testing.

[SSD]

That's basically the same thing that I suggested except that I believe alpha releases should be kept in house (LimeTech's servers only), and betas should take the place of what you describe as alphas (closed group testing only).  The RCs would then take the place of what you described as betas (wide-spread testing by the whole community, cue ingenious idiot effect).  As I understand it, that's the path that most software development takes.  Is there a problem with this method, or are we just arguing semantics?

 

I think it is a bit more than semantics - but maybe not a lot more. In this way, we are adding something (closed alpha testing) not taking away something (community beta testing).  It also addresses the issue that not every beta becomes a release candidate.  The terminology shift will help the community not feel they're being punished because of the actions of a few that loaded a beta they shouldn't have.  Telling people they're not going to be involved in alpha testing is not going to be a big deal.  Alpha implies a lot more risk than beta, and most people understand that.

 

I think it also defines the closed ALPHA period with very specific objectives (yet to be fully defined) and short timelines.  It really isn't BETA testing.  This may allay fears of extensive time commitments some have expressed about a closed beta test.

[Zen]

I think beta releases should remain open but the download page should be reorganized so that production releases are listed first, followed by any betas.  That way people will be making a deliberate choice to download a beta instead of blindly clicking on the first link they see.

[Rajahal]

That's basically the same thing that I suggested except that I believe alpha releases should be kept in house (LimeTech's servers only), and betas should take the place of what you describe as alphas (closed group testing only).  The RCs would then take the place of what you described as betas (wide-spread testing by the whole community, cue ingenious idiot effect).  As I understand it, that's the path that most software development takes.  Is there a problem with this method, or are we just arguing semantics?

 

I think it is a bit more than semantics - but maybe not a lot more. In this way, we are adding something (closed alpha testing) not taking away something (community beta testing).  It also addresses the issue that not every beta becomes a release candidate.  The terminology shift will help the community not feel they're being punished because of the actions of a few that loaded a beta they shouldn't have.  Telling people they're not going to be involved in alpha testing is not going to be a big deal.  Alpha implies a lot more risk than beta, and most people understand that.

 

I think it also defines the closed ALPHA period with very specific objectives (yet to be fully defined) and short timelines.  It really isn't BETA testing.  This may allay fears of extensive time commitments some have expressed about a closed beta test.

 

That makes sense.

 

I think beta releases should remain open but the download page should be reorganized so that production releases are listed first, followed by any betas.  That way people will be making a deliberate choice to download a beta instead of blindly clicking on the first link they see.

 

I agree 100%.

[dgaschk]

The beta link should have a strongly worded pop-up that requires the user to acknowledge that they are putting there data at risk.

[dyrewolfe]

Only provide the link to a beta in the announcement thread. That way those that want to try it will have to do a little work to get it. This will keep users from blindly downloading it.

[Joe L.]

Good idea. Wont stop all problems,but will make for better informed carelessness.

 

 

[PeterB]

better informed carelessness.

 

lol ... I like that!

[S80_UK]

Me too.  But "better informed carelessness" (tm Joe L) is exactly what will find the corner cases that meticulous planning sometimes misses.  It's not so different to the "ingenious idiot" referenced earlier.

 

Regarding alpha, beta and RC phases - going public with too many levels in my view increases the support effort for minimal gain and may only confuse some less tech-savvy users.  I would propose that alpha builds (likely to break since there are functionality changes) should remain internal to Lime Technology.  Betas accessible to a closed user group (may still break from time to time, signed up to with warnings, etc, but not necessarily requiring LT to approve members), and then RC's open to all, but again with warnings and a tick-box to have users acknowledge that they are responsible, not LT if things go wrong.

 

I support the earlier suggestions of making betas and RCs much less visible to the casual user.

[Whaler_99]

I can say, I tested and played with unRaid some time ago. When I recently came back to it as my choice for a new storage solution, on the downloads page I am presented with this shiny new "5 beta whatever" link that is "hot". Well golly gee wiz, I need to try that.

 

I guess I am a bit lucky, I have a lot of test gear, a bunch of experience with hardware, storage systems, all the terminology and such and some Linux knowledge. So playing with beta's for me is an informed choice and I know enough not to go clicking off into never nerver land if I don't know what I am doing.

 

All that being said, I think it needs to be a bit harder for new users, or less knowledgeable current users, to get their hands on a beta, and possibly lose everything. Remove it from the main site and post it through the annoucements forum. I think that will still get it all the exposure it needs, but you would have to consiously go looking for it at the same time...

 

Another $0.02 to this...

 

Shawn

 

[SSD]

Sometimes a ball pein hammer, and not a sledgehammer, is need to fix problems.  I like the idea of removing the betas from the Web Site, and putting scarier warnings around downloading them.  That might just do the trick!

[nia]

Sometimes a ball pein hammer, and not a sledgehammer, is need to fix problems.  I like the idea of removing the betas from the Web Site, and putting scarier warnings around downloading them.  That might just do the trick!

Exactly - I completely agree. This is one simple change (move the download behind several clicks and warning pages) and leave discussions, forums open, and Tom/LimeTech won't have to (waste) time administering  access and whanot for closed groups.

 

Simple - clean - effective. 

[treefour]

I like to test new stuff and am not overly afraid of screwing things up as that's how I seem to learn best. When I first found unraid I did exactly that, grabbed a beta and gave it a go. Being a complete Linux moron I quickly found out that, going that route was like trying to swim with and anchor tied to my foot. So I used a stable to start and have stayed with them as I learn.

 

I have since spent about 30 hours reading these forums figuring out all the little problems that have arisen with the server. As I'm sure you can see I have only posted once or twice as I have yet to find a problem that I couldn't figure out without info from the forum and the awesome the users here.

 

That being said I also think it should be a little harder for newbies like myself to access the betas, but not impossible, as I would be willing to give it a go once I was a bit more comfortable with my abilities/understanding.

 

Lastly I want to say thanks to all veteran users for all the time and effort they put in for the newbies like me.

[PeterB]

Remove it from the main site and post it through the annoucements forum.

 

I would suggest that the beta announcements should go in a new forum as part of the 'Version 5 Development' section, not in the existing announcements at the top of the board.

[lionelhutz]

I would suggest that the beta announcements should go in a new forum as part of the 'Version 5 Development' section, not in the existing announcements at the top of the board.

 

I like this suggestion. The 5.0 beta releases probably should be released in the 5.0 forum section.

 

Peter

[SSD]

I like all the announcements together. Wherever they are, people will come. Putting warnings around the downloads should be enough IMO. 

[NAS]

I am with bjp999.

 

Stop announcements.

Explain on the download page what the beta is and the REAL risks

Have all betas include a warning in emHTTP in big red letters NOT FOR PRODUCTION

Done

 

I would be strongly against a closed beta. It only hinders the project because of a few muppets that want new at any cost.

 

In some ways we as a community have to take some blame for this. For a long time we have been telling people to update to betas for certain real reasons but we should have been shouting that there was a real risk at the same time. Yes in almost every beta ever there has proven to be little or no risk but we should have said it anyway.

 

We must not end up in a situation where Limetech worries about releasing betas. Limetech should release early and release fast and let us do the bulk of the testing. Otherwise things will slow down again which harms the entire community

[PeterB]

Stop announcements.

 

I'm not sure about this.  This would slow the promulgation of the releases. Where would threads discussing the release go?  As soon as such discussion does start, any benefit of not announcing would be lost.

 

Have all betas include a warning in emHTTP in big red letters NOT FOR PRODUCTION

 

A warning has already been placed in the v5 releases - but, surely, this is too late ... in 5.06, the damage has already been done!

 

I would be strongly against a closed beta. It only hinders the project because of a few muppets that want new at any cost.

 

.

.

.

 

We must not end up in a situation where Limetech worries about releasing betas. Limetech should release early and release fast and let us do the bulk of the testing. Otherwise things will slow down again which harms the entire community

 

Agreed!

[SSD]

I am with bjp999.

 

Stop announcements.

 

I like all the announcements together. Wherever they are, people will come. Putting warnings around the downloads should be enough IMO. 

 

I think I was misquoted.