YANQ (Yet Another Noob Question(s))

[aethyr]

Another noob here with some questions...

 

I'm trying to decide the best approach for a fault-tolerant server that is least dependent on proprietary components as possible. The ability to easily recover in case of hardware failure is far more important to me than raw performance. For this reason, I won't use hardware raid because of the chance that the raid card could become obsolete.

 

I also won't buy any of the commercial, proprietary NAS out there for the same reason.

 

I want my file server, as an entity, to live forever. I may have to swap out parts. And over the years, piece by piece, it will probably be swapped out completely. But thats fine. A few hours of downtime to replace a CPU, mem, MB, HD is acceptable. But the server lives on indefinitely.

 

I am thus left with a few choices: open source software raid like freeNAS, semi open like unRAID and some more experimental solutions like ZFS+raidZ.

 

My concern with unRAID is that I am now software dependent on it. If for any reason heaven forbid, Tom gets gravely ill or dies, takes a long vacation or simply decides to move on, my file server can continue no more. As I understand it, even my USB key dies, can I still read/copy all my data from the drives? Or am I fubared?

 

While I understand the purpose of the serial key, its also a gigantic point of failure, no? Especially if it takes a while for a new key to arrive, or as mentioned before, no more keys will be produced. How have you mitigated this? Did you simply buy extra copies?

 

On a different note, assume I go the unRAID route - if I wanted to simply go JBOD in the beginning and add parity later, is that a simple procedure without reformatting? For example, I want to build my unRAID server with a SINGLE, new 250GB drive. 6 months later, I will add another comparable drive. Then 6 months later, I will add another. At this point 2 years later and 3 drives, I figure its time to add fault tolerance and will buy a 4th drive for parity. As I understand it, its just a simple matter of building the parity drive from the existing data, right? The reason I want to do this approach is to stagger drive's age and thus reduce the chance of 2 drives failing within a 24 hour period.

 

Finally, my concern is with the parity drive. As I understand it, parity must be the largest drive in the array. Does this  requirement become cumbersome to upgrading the capacity of the array? For example, if I want to increase capacity, I can't just replace an older data drive with the largest currently available drive because now that data drive is larger than the parity drive. So I have to upgrade the parity drive to the largest and make sure my data drive upgrade is equal or smaller...so most likely, I'll need to buy 2 drives. It seems like capacity upgrade is more costly than it appears?

[X4n]

i can tell you from personal experiance that ZFS+raidZ can also get corrupt, so it has the same problem as RAID

 

since your most definately going to setup a fileserver, the most easy thing for you to do is probably to test it for yourself (there is a free version of unraid)

 

about the to small parity drive thingie: just replace the parity drive with your new/larger drive, rebuild parity and the add your old parity drive as strorage to your server

[aethyr]

i can tell you from personal experiance that ZFS+raidZ can also get corrupt, so it has the same problem as RAID

 

since your most definately going to setup a fileserver, the most easy thing for you to do is probably to test it for yourself (there is a free version of unraid)

 

about the to small parity drive thingie: just replace the parity drive with your new/larger drive, rebuild parity and the add your old parity drive as strorage to your server

Can you elaborate on your ZFS experience/corruption?

 

Ok, so I want my parity drive to be as near, but slightly more than my largest data drive's capacity. Extra capacity on the parity drive is wasted, right? So if my largest data drive is 250GB, a 1TB parity = 750GB wasted?

[Joe L.]

Your parity drive must be as large OR larger than any data drive.  Yes, any excess capacity of your parity drive over any of your data drives is unused until you purchase a larger data drive.

 

The unRaid management utility is tied to the USB serial number (once you go over 3 drives)  The unRaid driver itself is baed on open-source and source code is included on the unRaid release.  If Tom were to stop development, somebody would take over and re-engineer a new management utility.  It almost came to that a few years ago when Tom took an extended leave of absence due to some personal matter.  I am not worried about the unRaid software... at very worst case I would be able to set up a three drive array with 4TB drives without a new license key (4TB don't exist today... but 1T drives did not a few years ago either)  Even if Tom were to stop his development, your existing USB drive will not stop working... it would continue to work.  Yes, you can purchase a spare... Tom's pricing encourages that option.

 

I do have a spare USB flash drive with a spare pro-key.  I'll probably never need to use it.  I purchased a pair of name-brand USB Flash drives and as long as I don't abuse them, odds are they will outlast me.

 

If you don't assign a parity drive, the unRaid array will act as your JBOD array... but even better, the user-shares feature will present your media collection as a single shared drive on your network.  So, if you do not want to protect your data, you can elect not to and add the parity drive later.

 

If you start with a pair of 250 Gig data drives, and then add a 250 Gig  parity drive you will be protected.  If you find a good sale on a new drive you can replace the 250 Gig parity drive with the larger drive and then re-allocate the old 250 Gig parity drive for data.  I personally did exactly that, but with a pair of 500 Gig drives initially. One for parity, one for data.  I've added drives since then and I'm about to swap out my parity drive with a new 750 gig drive since I got one on sale for about $150.

 

Over time (10 years from now) you will find it harder to find IDE based drives... I'd use SATA drives if building a new server.  The rest of the parts are so generic you will have no worries about dependency on hardware.  Odds are very high that Linux will support just about everything that will come along.

 

Your final concern is not an issue.  Today, if I were to have a drive fail I would purchase a drive based on the best value.  If 1TB drives were cheep, I'd get one of them as the replacement.  unRaid has a built in process that would allow me to assign my new 1TB drive in my parity slot, the old 500Gig drive in the failed data drive slot, and it would  move parity to the new drive and then rebuild the failed drive.  Note... when I refer to putting a drive in a slot, these assignments are logical, not physical... so even though my existing parity drive is an IDE based drive, I can use any disk on any controller, even an SATA controller for the new 1TB parity drive.

 

Is unRaid perfect... no... but it is better than just about anything else out there in terms of flexibility and ease of management for a home media server.    It uses standard hardware and file systems... any machine that can read a reiserfs can be used to get to the data in an emergency if your motherboard becomes toast. 

 

Joe L.

[aethyr]

Appreciate the detailed response, Joe.

 

If the thumb drive becomes toast, I can still mount the data drives on a different OS and migrate the files off right? So its not like a bad thumb drive will lock me out of my data?

 

Regarding JBOD mode: does this stripe files across the array or are files never stripped in unRAID ever? (BTW, I don't want striping - as I mentioned, I prefer recovery over performance)

 

user shares: how are files allocated across the array? So if I put a file on a user share, unRAID dies, I mount my data drives elsewhere on a non-unRAID OS, will I be able to locate and recover my files that were on the user share?

[Joe L.]

Appreciate the detailed response, Joe.

 

If the thumb drive becomes toast, I can still mount the data drives on a different OS and migrate the files off right? So its not like a bad thumb drive will lock me out of my data?

There are read-only drivers for MS-Windows... and of course, you could mount the drive in just about ANY Linux OS.  You could even use a "live-cd" to boot from and get to your data in read/write mode.

Regarding JBOD mode: does this stripe files across the array or are files never stripped in unRAID ever? (BTW, I don't want striping - as I mentioned, I prefer recovery over performance)

unRaid does not stripe data, ever.  Nor can any single file span multiple drives.  If you have a 4 gig ISO image, it must fit on a single drive.

user shares: how are files allocated across the array?

There are several allocation methods available.  You can choose how it allocates space and which drives are used.

So if I put a file on a user share, unRAID dies, I mount my data drives elsewhere on a non-unRAID OS, will I be able to locate and recover my files that were on the user share?

user-shares are NOT storage, but instead a system of smart links to the actual data on your physical disks.  It exists only in RAM, but to windows it looks like a huge folder, or set of folders for your data.  It is roughly equivalent to having a set of windows "shortcuts" to the actual files.  So, your files are never on the user-shares... but links to them are.  These links are created and maintained automatically for you...

 

Joe L.

[aethyr]

user-shares are NOT storage, but instead a system of smart links to the actual data on your physical disks.  It exists only in RAM, but to windows it looks like a huge folder, or set of folders for your data.  It is roughly equivalent to having a set of windows "shortcuts" to the actual files.  So, your files are never on the user-shares... but links to them are.  These links are created and maintained automatically for you...

 

Ok, looking at the user share documentation, it looks like a user share is a simple concatenation of the folders of each disk. So its hiding away the drive letter, more or less. So if I have UNIQUE_FOLDER on disk 1 with only 10MB free, under the user share, I'll see UNIQUE_FOLDER, attempt to write a 100MB file to it and fail?

 

Or, does unRAID replicate UNIQUE_FOLDER to another physical drive with more space and when I attempt to write my 100MB file, it will place it on the free drive? So its possible under UNIQUE_FOLDER, that files are scattered across the array, but each disk has a folder called UNIQUE_FOLDER?

 

So theoretically, if my array fails and I mount my drives elsewhere, I'd have to hunt and peck across my drives for all the files under UNIQUE_FOLDER. I'm not saying this is good or bad - I just want to know whats happening under the hood. I do see "split level" that allows me to control how the files get deployed across the array, but in practice, I think you'd want to minimize this sort of micromanaging.

[Joe L.]

From what I understand, it will create the UNIQUE_FOLDER on another drive if it needs to, as long as the user-share is defined to include the other disk (or disks)  You can restrict it to a single disk if you wish.  In the same way, you can write to the disk shares and control exactly where the files live.  You just need to stop the array and re-start it to have the user-shares get in sync. (for now... Tom has said it will be automatically updated in a future release)  This is how I have my array configured.  My "disk[1234] drives are exported read/write, but hidden.  My "Movies" user share is read-only, and visible on the LAN.

 

You get to define the disks used for any given user-share.  You are correct about the split level.  You can get to where it is an issue.  It was designed to keep a collection of affiliated files together. (a rip of a dvd into individual files, all within a single folder representing and named after the movie.)

 

Joe L.

[SSD]

Ok, so I want my parity drive to be as near, but slightly more than my largest data drive's capacity. Extra capacity on the parity drive is wasted, right? So if my largest data drive is 250GB, a 1TB parity = 750GB wasted?

 

I was in your shoes last month, trying to decide what to use for long-term file storage.  I came across unRAID and am now a very happy user and customer.

 

This issue of the "cost" of protection, that you raise here, is one of the reasons that I picked unRAID.  The fact that you need to dedicate ONE single drive to protecting all of your disks is a great advantage.  See the following example:

 

Say you create an array with 3 750G drives, you lose the capacity of one of the drives (cost of protection $150)

 

After you have moved some of your data to the array, you decide to move a 250G and a 400G drives to the array (incremental cost of protection $0).

 

Your array starts to fill up and you find a great deal on 2 more 750G drives (incremental cost of protection $0)

 

A year from now, 1T drives have dropped in price and you decide to add one to the array (your 750G becomes data and your 1T becomes parity) (incremental cost of protection 25% of the cost of the 1T drive)

 

You buy 1 more 1T drives and add it to the array (incremental cost of protection $0)

 

This is a pretty realistic pattern and exemplifies how easy unRAID is to grow.  Now clearly it would not be smart to have a single 1T drive protecting an array of 10 40G drives.  But with a little planning, unRAID is a very cost effective way to store your large media files.

 

I'll give you a final example using my own array.  I currently have 2x250G drives, 2x300G, 5x500G, and 4x750G (one is parity) drives in my array.  That is 5.6Tb usable (+ 750G parity).  Protection costs me $0.027 / gig (and falling as I add more storage).  Say instead that I put my 500G drives in one RAID array (5 disks), and my 750G drives in another RAID array (4 disks).  My usable space would be 4.25T, redundancy would be 1.25T.  The total cost of redundancy would be $250, which works out to $0.059 / gig (more than double).  My 1.1T in older 250/300G drives is sitting on a shelf.  And if I wanted to add a 1T drive, I'd be SOL without getting at least 2 of them, and then I'd be RAID 1 and lose HALF!  Add this to the fact that you are MUCH better protected from a dual failure scenario and you'll arrive at the same decision that I did - this is the right platform for long term media file storage.

 

- Brian

[aethyr]

Thanks guys for your responses...I think I'm convinced!

 

1 more question - can I do parity with just 2 disks or is 3 the minimum to have parity? I would think it should be allowed, although the parity drive would simply end up being a mirror. But I still want to do my incremental adding of hard drives, but with redundancy from the beginning. So I'd start with (1d+1p) then sometime later (1d+1d+1p) and so on.

 

Regarding array size...the larger the array gets the increasing probability that 2 drives will fail within the rebuild time. Does this make you guys somewhat nervous about building large arrays? I feel like a 14 drive array is asking for a 2 disk failure. It would be cool if unRAID could support adding a 2nd parity drive for max size arrays.

 

Regarding MB selection, as long USB boot is supported I should be fine, right? I'm getting an unRAID compatible third party ethernet card so I don't have to worry about onboard LAN compatibility with unRAID.

 

 

[Joe L.]

You can assign no parity drive, and have data drive(s) that is/are not protected from failure

or

You can assign a parity drive and a single data drive... as you said, because of "even" parity, the parity drive will be a mirror of the single data drive.

 

You can then expand the array as you find the need, and add data drives drives as you find them on sale.

 

You are correct, the more drives in an array, the more likely that two will fail at the same time.  Now, it is fairly un-likely to have two drives fail in a short period of time, so best bet is to replace any failed drive as soon as possible, otherwise you are playing against the odds.

 

If I had a 15 drive array and I had a drive fail I would simply purchase a replacement locally rather than look for the best price in a sale.  I can also shut down the array until I get a replacement.  Yes, it will take X hours to rebuild a failed drive. (depends on speed and size) I did a parity swap of a simulated failed 250 gig drive yesterday.  I put a new 750 gig drive in as parity and put the old 500 gig drive as a replacement for a (simulated) 250 gig drive failure.  The entire process took the better part of a day on my older IDE disk based array. 

 

I'd rather be at risk of a second drive failure for a single day out of several years than not have any protection at all.  In my case, I have all my original DVDs if I  need to rip them again.  If I lose a drive it is sad, but not critical.

 

If you have critical data, then back it up elsewhere... physically... otherwise a single fire/flood could fail all your drives at once.  unRaid is not a replacement for backups.

 

Joe L.

[SSD]

What he said!

[RobJ]

If you have critical data, then back it up elsewhere... physically... otherwise a single fire/flood could fail all your drives at once.  unRaid is not a replacement for backups.

 

Well said!  That cannot be stressed enough.

 

[aethyr]

What strategies are you using for backups? Is online storage a viable option or do you guys do traditional tape backups?

 

 

[Billped]

What strategies are you using for backups? Is online storage a viable option or do you guys do traditional tape backups?

 

 

 

I have a multi-tier backup strategy that is cheap and easy.  Maybe it will work for you, maybe not.

 

1. My most critical files are small - I am not a film-maker, for example.  For these files I backup over the net, mozy is one I use where I get 2GB for free.  That takes care of my tax files, resume, list of fixed assets, address book, and other critical documents.  The problem with "infinite" online backup is that the limiter is really your bandwidth.  Backing up 2GB takes a LONG time.  Backing up 10TB would take forever (OK, 1.5 years @2Mb/sec per my calc).

 

2. I use spare disk space on PCs throughout my house to store critical files.  So critical files on PC#1 may be on PC#2, those on PC#2 are on PC#3, and those on PC#3 are on PC#1.  Clearly I am not panicking about the security of those files because that means my kids can see my tax records.  They are well hidden and obscurely named, but perhaps they could find them.  Perhaps I can install truecrypt, but that would add more complexity than it is worth to me.

 

3. For my photos, I periodically burn DVDs and store them at work.  Two DVDs actually contain 100% of my digital photos.  Yeah, if I had a higher-res camera that wouldn't work as well, but for now, it does.

 

4. For music and videos, I have the original copies (I am not a big downloader or Netflix criminal) so the only loss is my time (valuable, but not invaluable).  Music files are small enough that I also move them around the house (50GB or so).

 

5. I have a couple of network connected drives, smaller ones, where I snapshot certain directories.  This is in addition to Unraid.

 

6.  I have unraid as a backup source for some files.

 

So, if the house burns down, I have my photos and tax records, all else is gone.  I can live with that.

 

If any one PC burns up, I'm protected.

 

If Unraid burns up, I have the original DVDs and CDs.

 

If I burn up, my wife is moving in with her boyfriend who can enjoy my unraid.  If anyone joins this board with my exact config, give 'em hell for me.   

 

 

Bill

[aethyr]

Those are some good ideas...

 

However, I'm extremely lazy and I'd have to find an automated solution or I'd never do it. I'm definitely thinking of a tiered approach as you - really important stuff gets backed up regularly to online storage somewhere. Semi important stuff, maybe gets put on tape on a weekly basis. And movies and such I could live with a loss.

 

By far the largest irreplaceable data are photos and videos of the family.

 

Here's an interesting question - does a HD have a shelf life? Meaning - if I filled a new HD with data. Unplugged it and stored it away for 50 years, will I be able to read my data from it? I may opt to just keep a collection of external HDs that I fill up once and squirrel away as permanent backup...

[SSD]

Check out this other thread:  http://lime-technology.com/forum/index.php?topic=1507.msg10144#msg10144

 

The last post by "LovingHDTV" seems like an interesting, automated solution!

 

I keep my important information on multiple PCs, but periodically cut a DVD of digital pictures and give to my sister.  She lives about 1/2 hour away.

[Joe L.]

By far the largest irreplaceable data are photos and videos of the family.

True... if my tax records were destroyed, somehow, I'm guessing I'll still need to pay taxes. 

Here's an interesting question - does a HD have a shelf life? Meaning - if I filled a new HD with data. Unplugged it and stored it away for 50 years, will I be able to read my data from it? I may opt to just keep a collection of external HDs that I fill up once and squirrel away as permanent backup...

Unfortunately, everything has a "shelf life."  Even if the hard disk was fine, try finding a computer to connect it to.  As an example, try finding a computer with a 8 inch floppy disk drive. (yes, I said 8 inch... common on very early CPM based machines)  Or today, finding a computer with a 5 1/4 inch floppy... they're pretty rare... in fact, finding ANY floppy drive on a recent PC.

 

Today, probably best bet is probably an external USB drive.  You can buy a 1TB one for under $300 on sale.  Copy your most irreplaceable data there and store the drive elsewhere... (possibly in bank safe deposit box)  As technology evolves, every five years or so, move the data to current media.  I can't guess what storage will be like in 50 years, but I'll bet the IDE, SATA, and USB buss drives will be long since made obsolete.

 

Joe L.

[SSD]

I have found that hard disks that sit around for a years start to have sectors go bad.  The same may happen if they are in a PC, but in a PC their "smart" monitoring will cause remapping to occur and usually prevent data loss.

 

If you are going to use a hard disk (external or internal) for long term backup, I'd recommend looking at "quickpar".  It is a tool that will create recovery files that you can use if one of the files in a protected set is damaged.  It is a very efficient way to protect a lot of files from slight to moderate corruption with little space.  (It will also tell you if you HAVE corruption.)  I've started to do this even with backups I do to (data) DVDs.

[WeeboTech]

What strategies are you using for backups? Is online storage a viable option or do you guys do traditional tape backups?

 

Ok, Here's the funny thing.

unRaid IS my backup strategy.

 

For my most important files, I use RAID1 on the servers that require it.

 

My music for example. (I run an internet radiostation).

The music repository is software raid1 because I need it up as much as possible.

I then RSYNC this to the unRaid server to another spindle. WaLa.. Backup.

 

I use these trayless SATA holders.

When needed I swap out a drive, Rebuild,  put the other drive away and I'm done.

So in reality I use hard drives like tapes.

 

For my movies, I have a readyNAS in X-Raid.

My plan is to start rsyncing the files over to unRaid and use that as a backup.

When price permits, I'll remove drives, and put one away and put a new one in it's place then remove the files from the ReadyNAS.

 

All of my unix machines use something called HDUP to back up their files.

I have it deposit the archives onto the unRaid server. 

I used to use a lil Kuro box..(20watts usage) but have since migrated to unRaid.

Once a month. I swap out the drive.

 

However, I just built a new separate 3 drive backup server.

This one is set to power up at a certain time (from the bios), trigger all the backups, then shut down completely thereby saving even more electricity.

I plan to switch to Bacula http://www.bacula.org/en/ Then as I need space, just add more disks and let the file storage daemon worry about where to put stuff. this will allow my laptops and windows workstations to be backed up too.

 

 

 

[aethyr]

I have found that hard disks that sit around for a years start to have sectors go bad.  The same may happen if they are in a PC, but in a PC their "smart" monitoring will cause remapping to occur and usually prevent data loss.

 

If you are going to use a hard disk (external or internal) for long term backup, I'd recommend looking at "quickpar".  It is a tool that will create recovery files that you can use if one of the files in a protected set is damaged.  It is a very efficient way to protect a lot of files from slight to moderate corruption with little space.  (It will also tell you if you HAVE corruption.)  I've started to do this even with backups I do to (data) DVDs.

 

This is what I'm afraid of...I recognize quickpar once I went to the homepage for it - its used in newsgroup binaries to rebuild  RARs with missing files. Sounds promising. I wonder what the degradation rate is for HDs. PAR2 is pretty forgiving, but I believe with over 10% data corruption/missing, it starts to have trouble reconstructing the original.

 

LovingHDTV solution seems like he built an unRAID server at his brother's house and rsyncs from his unraid to his brother's over the wire. This is similar to using an online backup service, but suffers from bandwidth limitations.

 

Of course, another option on the horizon is blu-ray burners. Granted, they're only 25GB, but with double layer or more they could be viable...that is until you start ripping blu-ray movies to unRAID 

[SSD]

PAR needs a good recovery block for each bad block.  If you have 10% pars you can recover from almost a 10% failure.  If you have 20% pars you can recover from almost 20% failure. etc.