Jump to content
jonp

Extra redundancy

69 posts in this topic Last Reply

Recommended Posts

I also would pay extra...

 

-1 I am not sure if I would pay extra but if there was a target amount of money for the project (i.e. need 5k to develop and accept donations towards that end) and if goal reached then development would begin or monies refunded I would donate (subject to having a commitment to when it would be delivered).

 

I paid for the Pro version up front and I don't think it is a good precedent to accept that you will pay for new features when you already have the top licensed version! But thats just me.

 

My issue with Unraid (as I refer to above) at the moment is not the software - it is the management of the releases. For I find it hard to know when things are going to happen. There is a lack of communication / commitment on release schedules etc. This might make me pause on donation too. Whats the point of paying for something (or donating) when you have no idea when it is going to happen. Especially if it is going to be a long time (e.g. > Year) - you might not need it then!?

 

http://lime-technology.com/forum/index.php?board=63.0

 

As a Project Manager myself for many years - the very minimum (even in Initiation pre Planning when the scope and schedule is not fully developed)is high level milestones and dates. Unless there is some information buried in the forum that I cannot find - there is nothing!?

Share this post


Link to post

Jono just relplied to my earlier thread addressing up and coming changes relating to schedule information.

 

On that basis id certainly donate to an extra redundancy initiative.

 

As I said before, I think the company should consider a seperately funded "Project" to do this. Highlight the funds required and request donations. Perhaps even donation Milestones- e.g Milestone for exisiting team, Milestone where 1 person contracted in to help Milestone 3 where 2 people etc

 

The more donated and earned then that's the project that is initiated.

 

Some safety net for investors / donators too. If at least milestone 1 is not reached all get refunded (Maybe minus 1% for costs).

 

Just thinking out loud!

 

EDIT: Just had a look at GoFundMe that Weebo suggested. No restrictions for commercial entities running fund raising initiatives. Flat 5% fee. No withdrawal fees. I'd be interested to see in what currency it is run or if it is selectable or pre agreed exchange rates etc etc I'd also consider withdrawing regularly as it is no fee. Could be viable. You could also ask via these forums for donators to add the 5% and post those donations on the forum too!

Share this post


Link to post

Many ISV's don't charge for new versions, but occasionally something comes along which nobody had envisioned and is a major step-change, it often means that the development effort needed to hit it is substantial and the extra functionality it brings is significant.  It's not unusual for an ISV to charge for this and sometimes not everybody takes it up, although usually, they do over time.

 

Unless unRAID is developed with a reusable component based approach, the effort needed might be nothing short of a rewrite.  In any event and without seeing the way it's put together, it would appear the core disk handling system would need a rewrite e.g. writing data, initialing disks, parity swap, disk rebuild, parity rebuild, disk expansion etc etc.  At the same time, if you're doing all that, it would make sense to increase the number of disks supported, not so much for home enthusiasts, but to open the door to commercial opportunities.

 

IMHO, one of the most important USP's that unRAID has, is a file system per disk. 

 

I understand the viewpoint that people have paid for PRO, but I suspect all this extra functionality would lead to a new category (and it should).

 

Technology is often "enabling".  You could say, why support more than 25 disks because it's difficult to build a box to hold them.  Once you can support a lot more, people will find a way and they will find commercial uses for the resulting systems.  Disks are cheap, storage systems are not, ask EMC.

 

Occasionally, someone comes along and says "I can do that better and cheaper".  I am reminded of an ISV that was the market leader in it's sector selling it's software packages on DEC PDP-11 mini-computers that cost typically $100k.  They groaned with 20 or 30 users on them and the month-end routines took a weekend.  They had a very large, vocal, unhappy user base and sales stalled.  A guy in his early 30's came along and bought the company, ran the code through a converter and produced 'C', compiled it for SCO UNIX and built 486DX2-66 computers in-house in mini-computer sized, expensive looking cases imported specially from China (where else), adding ethernet terminal servers and suddenly the software could support a 100 users and that month-end routine that used to take 48 hours, completed in less than 30 minutes.  Those cases were 95% empty, but they looked a million dollars.  They cleaned up selling hardware costing less than $10k for >$100k, and a new category of software licence to people who had already paid for their "PRO" licences.  Within a year he was coming to work in a new Rolls Royce and was soon gone to his next project.

 

Sometimes you have to look at whats out there and think "if I do this, and that, I can do it better and cheaper".  When I first saw unRAID my mind raced.  It was because it could support so many drives, inexpensively and had a file system per disk.  Those are it's major USP's.  It's major limitations are one array per implementation, single parity, the number of disks it supports and the credibility of the company to large corporates.  In particular, single parity on 20 disks is just not credible, you WILL lose some data eventually.  It's no good saying backup is the way to go, I see backup/nls as the major use for unRAID in the commercial world.

 

It does seem a lot of people want multiple parity disks and that user base might fund opening doors into the commercial world for limetech.  I have been retired 8 years now and I don't drive a Rolls Royce anymore, but I would like to see Tom driving one.  I would be happy to contribute a couple of hundred bucks for multiple parity.  With the right strategy, some businesses will take a risk on a small supplier.

 

 

Share this post


Link to post

With 6.0 finally coming out, can we please schedule this for 6.1? I'm honestly shocked this is still unscheduled (does Lime Tech not have the capable programmers?), but I'm assuming most average users aren't aware of how big a deal this really is

 

My drives are getting old, and I don't trust a single parity system anymore, especially not with the amount of drives and data I have. I just bought some new drives to replace the dying ones, but the system to offload data and swap in and out drives is so cumbersome and/or dangerous, that I'm setting up a second, dual parity system based on FlexRAID's tRAID just to copy everything safely

Share this post


Link to post

With 6.0 finally coming out, can we please schedule this for 6.1? I'm honestly shocked this is still unscheduled (does Lime Tech not have the capable programmers?), but I'm assuming most average users aren't aware of how big a deal this really is

 

My drives are getting old, and I don't trust a single parity system anymore, especially not with the amount of drives and data I have. I just bought some new drives to replace the dying ones, but the system to offload data and swap in and out drives is so cumbersome and/or dangerous, that I'm setting up a second, dual parity system based on FlexRAID's tRAID just to copy everything safely

 

I have a feeling this is going to be more like a 7.x feature rather than a 6.x.

 

Still no clear roadmap though which continues to be disappointing.

Share this post


Link to post

In the last two weeks, two old drives went defective. I had luck that I could rescue nearly all data, but if the two drives would have been failed at the same moment, that would have been it.

I really appreciate if that feature could be implemented soon! If you still have concernings about the priority, just watch the view count of the feature request threads wanting multiple parity compared to the view counts of the other feature requests. This thread has the largest view count of all feature request threads in this subforum of unsheduled features. We really need it!

Share this post


Link to post

It's fairly clear this is not a high-priority item from Limetech's perspective.  Unless, of course, you're ready to order a very large # of servers and demand this be included  :)

 

From

... Implementation is going to be a complex task which we cannot undertake until some other features are addressed (or unless someone comes to us and says, "hey we'll order 1000 servers but only if you have P+Q parity support."

 

However, regardless of that, remember that a fault-tolerant system is NOT a substitute for backing up your important data.  Fault-tolerance is to keep your system running ... not to back it up.

 

The easiest way to get both additional fault-tolerance AND ensure your data is well backed up is to build a back-up server.    That's what I (and quite a few others) do.    I simply have the primary server automatically sync to the backup every night.    This not only provides a current backup, but provides fault-tolerance for up to 3 disk failures with no data loss.  [My backup server actually backs up two other UnRAID servers, but the concept is the same.]

 

 

Share this post


Link to post

Note that some folks backup their data on the SAME UnRAID server by simply having one (or more) dedicated backup disks and a "Backups" share.    They then backup their important data to that disk (or share).

 

This is better than nothing ... but doesn't improve the redundancy.  Clearly you're still in a position where a 2-disk failure could result in lost data if the two disks that went bad were the one with your important data and the backup disk that was "backing it up".    This is somewhat akin to creating a 2nd partition on your hard drive and backing up the data from the primary partition to that 2nd partition ... if the drive fails, you're still "toast."  It does, of course, provide some of the benefits of a backup -- ability to recover files that have been corrupted or inadvertently modified or deleted -- but misses one of the key elements ... providing the ability to recover your data in the event of a catastrophic system failure.

 

Share this post


Link to post

You are right concering backup and my important data IS backuped on several disks and in a private cloud (so in two different locations).

But nevertheless, a full backup of the "less important data" from up to 24 devices in an unRAID is too costly and not worth it - but a second (or more) parity drives protects from two (or more) disk failures.

 

I am not persisting in saying that without evidence!

See this article that talks about the assumption, that two disk parity will not be enough by 2018 - single parity is not enough for 20 years already:

http://queue.acm.org/detail.cfm?id=1670144

There are more comparisons and mathematical calculcations that prove the urgence of multiple parity devices for having correct data. We do not talk about backups but having "working, running and reliable storage systems".

 

Double parity would be good - papers for the mathematics and algorithms can be found here:

https://www.kernel.org/pub/linux/kernel/people/hpa/raid6.pdf

http://web.eecs.utk.edu/~plank/plank/papers/CS-96-332.pdf

So it is not a secret. But multiple parity would be perfect as it would be most flexible.

 

I hope Limetech reconsideres the priority. But I cannot order 1000 licences - I ordered two - the lack of multiple parity still has a bad aftertaste.

Share this post


Link to post

While I understand the arguments, and agree that LT needs to make a double parity system (or more), the one flaw in the arguments is that an URE is not a disk failure.  unRaid during reconstruction will not fail because of a URE, or multiple UREs from multiple drives.  You may wind up with a corrupted file (or none at all depending upon where on the disk the URE happened).

 

Nothing is ever going to replace a solid backup plan, and myself I keep MD5 checksums of ALL files on my servers and periodically check all files against them, and always check the files on a rebuilt drive to look for possible corruption.

Share this post


Link to post

... the one flaw in the arguments is that an URE is not a disk failure.

 

It may not cause a red-balled disk; but it's certainly a failure in the sense that it results in an incorrectly rebuilt disk.

 

 

... unRaid during reconstruction will not fail because of a URE, or multiple UREs from multiple drives.

 

Wasn't aware of that ... never had an error during a rebuild.    But it's nice to know -- and as long as you have a way of confirming which files have been corrupted as a result, it's good that the rebuild continues.

 

 

... I keep MD5 checksums of ALL files on my servers and periodically check ...

 

Ditto.  I am VERY thoroughly backed up ... a complete backup of both of my servers on my 3rd (backup) server; PLUS a complete copy of my media server on an additional set of disks that are stored in a fireproof, waterproof, data-rated safe.    In addition, all personal data (photos; documents; financial and other records; receipts; music; etc. is all stored in at least 4 places (my computer; wife's computer; "misc" UnRAID server; and the backup server) PLUS is backed up to the cloud (via Carbonite).

 

In addition, as noted, I have MD5's of all the data on my servers AND on all of my backup disks in the safe, so I can easily verify any given set of data with a single right-click, "verify checksums".

 

 

Share this post


Link to post

I feel this thread has gone slightly off topic. Such is the thread of these conversations - we end up always talking about backup. Well I want to re-emphasise why I would like LT to prioritise this:

 

1. My use of increasingly larger size drives (currently 8TB but will increase)

2. My use of an increasingly large amount of drives (currently 6 but will increase)

3. My need for continued availability

 

Id like the parity to be scalable n+ x but I'd settle for n+2 initially.

 

So if this feature was available now - in my case id have 7 total drives (1 additional parity) and I'd feel much better and it would be a better setup.

 

In essence my reasoning is because it takes longer to rebuild larger drives when a drive fails, so i have a longer window of exposure if something happens. Plus with an increased number of drives I am increasing my likelihood of a failure during a rebuild.

 

Either way by having that second parity drive, it protects me from an additional drive failure during the rebuilding I will eventually have to make.

 

If I do get a failure in my current setup and then another ill have to spend days restoring my main server. That is not a scenario I want.

 

I know I'm probably opening the door for those to say the likelihood of a second drive failure is low or some other BUT at the end of the day if it can happen. It will. That's why I feel LT should prioritise this feature.

 

I am not concerned about my data. I am completely backed up both on and off site.

Share this post


Link to post

... I know I'm probably opening the door for those to say the likelihood of a second drive failure is low or some other BUT at the end of the day if it can happen. It will. That's why I feel LT should prioritise this feature.

 

Actually the likelihood of a second drive failure is fairly high with the very high-capacity systems folks are building these days.    Typical consumer-grade hard drives have typical uncorrectable bit error rates of 1 in 10^14 bits.  [Enterprise class drives are an order of magnitude better -- 1 in 10^15]    Consider that a single 8TB drive contains 64Tb = 64 x 10^12 = 0.64 x 10^14 bits ... so from a pure math perspective there's a 64% chance of an unrecoverable error if you write the entire drive and then read it back !    Fortunately, real world experiences by large data storage enterprises have shown that reality is that the drives are appreciably more reliable than those specifications imply.    Some have speculated that the 1 in 10^15th rate advertised for Enterprise class drives is more accurate, even for consumer class drives ... and thus the error rates are actually 1/10th as bad as the math indicates.    Others have noted that even that is often exceeded -- but the drive manufacturers are very cautious in what they claim, so they don't claim the higher reliability that modern drives can actually achieve.    But in any event, with multi-TB systems, the odds are indeed getting high enough that a 2nd level of fault tolerance would be VERY nice to have  :)

 

 

I am not concerned about my data. I am completely backed up both on and off site.

 

Excellent.  You'd perhaps be surprised how many folks go to the expense of acquiring large amounts of data; build a fault-tolerant system to store it; and don't bother to back it up.

 

Share this post


Link to post

See THIS POST

 

In summary, critical data must be backed up. No question. But a large media server can be recovered "in other ways". There may be a cost to that recovery, maybe a high cost including time and effort, but when dealing the Media, which are NOT unique works, the reliability of the primary array definitely comes into play.

 

Taken to absurdity, if the data would cost $1 to reconstruct, and the backup solution would cost $5000, regardless of the chance of loss, there is no need to backup the data.

 

At the other end, if the data would cost $1M to reconstruct, and the backup costs $5000, with a 10% chance of data loss, there is an absolute need for backup.

 

But what about ... if the data would cost $5000 to reconstruct, and the backup costs $1500, with a 10% chance of data loss

 

vs

 

if the data would cost $5000 to reconstruct, and the backup costs $1500, with a 0.1% chance of data loss.

 

It may be quite economic for a users to be willing to pay an additional $300 for another drive for dual parity protection, that significantly reduces the chance of failure due to disk-related issues, than it is to pay $5000 and double the cost of each drive they buy in future to have mirrored systems.

 

So I don't think it is fair to say that the extra redundancy is ONLY for high availability. It can, for some users, alter the equation for how much data to backup.

 

The other point, that seems to have not been made at least not recently here, is that the extra redundancy would very likely help pinpoint the source of a parity issue. Today a parity error is assumed to be a problem with parity. But with dual parity, that parity issue could be isolated to a specific disk, and the correction applied correctly.

 

Dual parity / extra redundancy checks a very important box on the unRAID feature sheet, but is also very valuable to many users IMO.

Share this post


Link to post

I don't think that at least double parity is too difficult, I would recommend the diagonal parity algorithm described in this paper:

https://www.usenix.org/legacy/events/fast04/tech/corbett/corbett.pdf

 

It is easy to implement, would be fully compatible with the underlying unRAID design (XOR pattern) and does allow for double parity (which would be a lot better than the single parity as we already found out, especially when we have up to 24 disks in the array).

 

That would resemble RAID-DP, described here:

http://www.netapp.com/us/media/tr-3298.pdf

 

There is also a nice example in the latest paper, that shows how to reconstruct using diagonal parity. (But that is only an example, the right mathematical way is described in the paper above - also mentioned in the latter paper!)

 

Moreover, double parity disks allows for identifying a bit error. With a single parity (like now), when a bit error occurs, the error cannot be identified on which disk it came from, so the parity disk (during parity sync) just is updated and the original data is lost forever. If that bit error occured during a filesystem header, we have a problem. Double parity can identify the bit error by combining the horizontal and diagonal information for identifying the location of the error. That would allow unRAID to be reliable as the data can be verified (would call that a "verify sync" or so, not a parity sync anymore). Different disk sizes shouldn't be a problem, too, as one can fill up with "simulated zeros" when there is no more disk data - like in horizontal parity done already.

 

Before Limetech is searching for an ultimate solution that would cost license information, that diagonal parity could be a very fine solution for "now" and the next years.

As double parity would allow for integrated, verified data and for having another fault tolerance (especially when rebuilding the array when a disk was faulty) and as the algorithm is public available in the first paper, I don't see any reason - especially as it uses XOR parity like already used in unRAID - why it shouldn't be possible to implement that.

 

And for the users who do not want to use the double parity, the double parity disk can just be switched of and unRAID works like it works now. The second parity is really optional and can be added at any point when using diagonal parity. So it is not like in RAID 6 that one has to choose how many parity disks one wants at the beginning but in this case, one can add the second parity when one is ready for it.

 

I hope I did not miss anything, but I do not want to moan without proposing solutions. This is my solution for having double parity with unRAID - I think it is worth it.

Share this post


Link to post

I think it's worthwhile mentioning that in this thread, and I think in other threads multiple people including myself have expressed a willingness to pay for dual parity.  If it's a time consuming or difficult feature to add, as has been suggested, I don't mind paying for it to make it worth Limetech's effort. 

 

Thankfully I've never had 2 drives fail at one time, but I have had single drive failures a couple of times, and the panic of realization that if another drive happens to fail before I can replace the failed drive is horrible. 

 

And yes I know that backups are the best solution to this problem, but as others have mentioned do I put out thousands of dollars on putting together a backup server, or do I just spend a couple of hundred dollars on another parity drive.  And yes I've heard the arguments where people ask "How much is your data worth".  But frankly that math changes dependent on how much money you make and what your other financial obligations are. 

 

The other thought that's crossed my mind a few times recently is having a vm unraid backup server running through kvm on the bare metal unraid.  This would still be an expensive solution, but maybe not as expensive as putting together a whole new system.  Thoughts???

Share this post


Link to post

... The other thought that's crossed my mind a few times recently is having a vm unraid backup server running through kvm on the bare metal unraid.  This would still be an expensive solution, but maybe not as expensive as putting together a whole new system.  Thoughts???

 

That would work if you have space for the extra disks and the necessary SATA ports.  Backing up on the SAME system isn't as good an idea as having independent backups, but it's certainly better than not backing up at all.

 

An even more economical way is to simply not backup to another system -- but to a set of backup drives.  Many of us have a bunch of older drives -- perhaps as you've upgraded to larger drives; replaced drives that were getting older (perhaps had a bunch of reallocated sectors); etc. => these work just fine for backups.  You just need to get in the habit of copying all of your new media to both the server and the "current" backup drive (replace as they fill up) ... and just store the backup drive in a secure place.  I use WiebeTech DriveBoxes and store them in a fireproof, waterproof data-rated safe.    I also create checksums for everything before I copy it to the server and backup drives, so it's trivial to verify files on either the server or the backup drives.

 

http://www.amazon.com/DriveBox-3851-0000-11-Hard-Disk-Case/dp/B004UALLPE/ref=sr_1_1?ie=UTF8&qid=1433799538&sr=8-1&keywords=wiebetech+hard+drive+case&pebp=1433799541357&perid=1QPPRNBS6ZSGBGXV7NRQ

 

[i've done this for well over a decade and probably haven't bought more than 3-4 drives for backups ... all of my other backup drives were simply re-purposed drives]

 

I DO happen to have yet-another complete backup on a backup server, but that's not really necessary if you already have a complete set of backups  [i was building a backup server for my 2nd UnRAID server and just decided to make it big enough to also backup my primary media server.].

 

Share this post


Link to post

If this is implemented couldn't you mirror them for better performance, say if you are hosting s media server and the same file is accessed by multiple users

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.