Kryspy Posted January 9, 2014 Share Posted January 9, 2014 What is the work involved in creating an overlay type of product with the unRAID MD system? Kryspy Quote Link to comment
ironicbadger Posted January 11, 2014 Share Posted January 11, 2014 Lolz. Ask tom. Sent from my iPad using Tapatalk Quote Link to comment
grumpybutfun Posted January 11, 2014 Share Posted January 11, 2014 unRAID = Resier on top of RAID 4 using modified md module. Run BTRFS on top of unmodified RAID 4 md module if you want parity on one drive. Or run BTRFS on top of RAID 5 or 6 if you want single / dual parity across all the drivers. Quote Link to comment
vl1969 Posted January 11, 2014 Share Posted January 11, 2014 unRAID = Resier on top of RAID 4 using modified md module. Run BTRFS on top of unmodified RAID 4 md module if you want parity on one drive. Or run BTRFS on top of RAID 5 or 6 if you want single / dual parity across all the drivers. Hay, with btrfs you don't need to run it on top of raid5 btrfs has that built in Sent from my SGH-T889 using Tapatalk Quote Link to comment
grumpybutfun Posted January 12, 2014 Share Posted January 12, 2014 Hay, with btrfs you don't need to run it on top of raid5 btrfs has that built in BTRFS with RAID 5/6/10 etc. is not stable yet. BTRFS without RAID is. That is what I suggested running RAID 4/5/6 (md) with a giant BTRS File System (none RAID) on top. Quote Link to comment
Kryspy Posted January 12, 2014 Author Share Posted January 12, 2014 Got it. Been doing some reading up on BTRFS but I think it is more than I need out of a media server at the moment. EXT4 with SnapRAID and AUFS is doing just fine. Kryspy Quote Link to comment
abs0lut.zer0 Posted January 12, 2014 Share Posted January 12, 2014 does BTRFS function more like ZFS where you need drives same size in pool sets or does it work more like unRAID where different sizes can be used and the parity must just be biggest ? Quote Link to comment
grumpybutfun Posted January 12, 2014 Share Posted January 12, 2014 does BTRFS function more like ZFS where you need drives same size in pool sets or does it work more like unRAID where different sizes can be used and the parity must just be biggest ? There are not many RAID 4 "products" out there and I have never worked for or consulted at a company who used it. The question you should be asking... Why is that? It's not to say that you will not like the answer you come up with it. It's just that you are trying to compare apples / oranges and I do not think you know the differences, pros / cons, risks / rewards or whether or not unRAID is the apple or orange. Common Mistakes people make when comparing unRAID with ZFS 1. Think only in RAID and RAID 4 at that. 2. Think you can only have one "RAID". 3. Their wallet has a lot of say / influence and are okay accepting more "risk" to save a buck. ZFS for Dummies 1. A zpool is one or more VDevs allocated together. 2. You can add more VDevs to a zpool after it is created. 3. Think of zpools as as volumes. 4. Hard drive(s) goes inside VDevs. 5. Vdevs go inside zpools. 6. Zpools store your data. 7. Disk failure isn’t the concern with ZFS. Vdev failure is! Keep the VDevs healthy and your data is safe. 8. Vdev with RAIDZ2 provides redundancy against any 2 simultaneous hard disk failures. Any 2 hard disks can fail with no loss of data. 9. You can sleep a zpool. Now here is where EVERYONE loses their MIND... 1. Hard disks put in a VDev cannot be removed. Only replaced and only if you have enough redundancy to support the disk replacement. 2. VDevs added to a zpool cannot be removed under any circumstances. ZFS allows for a zpool to expand in only two ways. Option 1: Add additional VDevs. Option 2: Replace all of the hard disks in a VDev with larger hard drives. Note: Replacing all of the drives in a VDev will increase available storage space. This method is useful if you want to buy 1 new disk a month. Keep in mind that the zpool will not grow until you have replaced all of the disks in a VDev. What you SHOULD do instead... Most rookies / noobs think in RAID and think you can / should have one MASSIVE zpool with EVERY SINGLE DRIVE IN THEIR SYSTEM IN IT. WHY? I have no idea. On one of my old production servers I have 3 VDevs for 3 different zpools. I could lose 6 drives at once (2 in each VDev) and still not lose any data. I could of added my last VDev to one of my zpools but didn't because I don't want to have to babysit 2 VDevs in 1 Massive zpool knowing that x.265 (shrink all your videos by 30% - 40% maybe more) rolling out over the next couple of years. When either of my three zpools (each has their own VDevs) start getting close to being full... Option 1 - I simple create ANOTHER vdev and create a forth zpool. Option 2 - Assuming it's my original VDev that gets full first (old 1 TB drives)... Replace those old drives that need to be replaced soon anyway with larger 4 TB ones. Option 3 - Create ANOTHER VDev / zpool but spend a little extra so I have room for growth. Also to have the ability to copy all my stuff from my original vdev / zpool (the one I mentioned above) and remove the old udev / zpool from my system. Leaving me with 2 zpools (each with their VDevs). Where people screw up with ZFS... P oor Planning, want to compensate / show off to the neighborhood and create a MASSIVE zpool with 1 udev and all 20 drives in their system. THAT is not what most users should do and it will make your life MISERABLE down the road. When I started out... I had 1 zpool and 1 VDev with all my TV Shows, DVDs, Movies. When my library grew... I created another VDev and zpool and moved Movies to it (DVDs, Movies and Blurays rips). When my library grew some more... I created another VDev and zpool with only my Blurays rips So now... zpool0 - TV Shows zpool1 - DVDs / Movies zpool2 - Bluray Rips If I am not downloading, watching or ripping something all 3 sleep. If I watch a TV Show... only zpool0 (and those drives in that VDev) wake up. If you understand any of what I wrote / explained above... Then you have to factor in the File System features of ZFS / BTRFS too. Checksums in Metadata for Data Integrity / Self-healing / Repair and Recovery Tools / Security / Snapshots / Copy on Write / Compression / etc. but that is a topic for another day. Quote Link to comment
Harpz Posted January 12, 2014 Share Posted January 12, 2014 That sounds pretty dam cool, I especially like the ideas of the 3 separate pools and the ability to loose up to 6 disks. Wow. What's the minimum disk requirement for each pool? Could you have 3 pools of 5 disks for example? Don't all the disks in the pool all need to be the same size? Instead of me asking loads of questions here do you have any links an idiot like me can follow and learn from, this sounds very very interesting? Quote Link to comment
grumpybutfun Posted January 12, 2014 Share Posted January 12, 2014 That sounds pretty dam cool, I especially like the ideas of the 3 separate pools and the ability to loose up to 6 disks. Wow. That made sense to you what I wrote? You must be smart. Most people can't understand it. Links? There are a bunch out there. ZFS combines a file System (which is amazing) and RAID into one. My advice is to focus 100% on the features of the FS and once you understand those then circle back around to the RAID part later. Otherwise your brain will get scrambled. Quote Link to comment
Harpz Posted January 12, 2014 Share Posted January 12, 2014 I thought I understood it or at least the concept of how it works was looking around and came across raidz2 so would need 3 +2 disks, so 3 pools of the same config would give you the 6 disk redundacy. This is about as far as it goes, Is the space on the 3 data disk the only part that's accessible or is all the space available, ie 3x2tb or 5x2tb? My Linux experiences consist of ubuntu/xubuntu and a little snapraid on ubuntu server, this had webadmin installed so understand bits of that also, never be afraid to "get my hands dirty" though. I need to do allot more reading / playing it would seem, lol. Roll on unraid distro edition Quote Link to comment
Kryspy Posted January 12, 2014 Author Share Posted January 12, 2014 Hey, even the Messiah of Morons followed it Of course I've read up on ZFS before. Harpz, If you find ZFS imprerssive, give BTRFS a light read. It has most of the features of ZFS but with greater flexibility for pool creation/modification. http://blog.osnexus.com/2013/04/24/btrfs-zfs-the-good-the-bad-and-some-differences/ http://en.wikipedia.org/wiki/Btrfs Kryspy Quote Link to comment
grumpybutfun Posted January 12, 2014 Share Posted January 12, 2014 Hey, even the Messiah of Morons followed it Of course I've read up on ZFS before. You have a good sense of humor and don't steal my line. I was talking to everyone else... I knew that you were a pro on ZFS. If you find ZFS imprerssive, give BTRFS a light read. It has most of the features of ZFS but with greater flexibility for pool creation/modification. I agree. Just waiting for RAID 5/6 to be stable in BTRFS. Quote Link to comment
grumpybutfun Posted January 12, 2014 Share Posted January 12, 2014 I thought I understood it or at least the concept of how it works was looking around and came across raidz2 so would need 3 +2 disks, so 3 pools of the same config would give you the 6 disk redundacy. This is about as far as it goes, Is the space on the 3 data disk the only part that's accessible or is all the space available, ie 3x2tb or 5x2tb? My Linux experiences consist of ubuntu/xubuntu and a little snapraid on ubuntu server, this had webadmin installed so understand bits of that also, never be afraid to "get my hands dirty" though. I need to do allot more reading / playing it would seem, lol. A lot of users think with their wallet and are fine with single parity. In unRAID we don't have a choice. Where as in Linux or ZFS you have Single, Dual or Triple parity if you want. Different strokes for Different folks as they say. Quote Link to comment
drawz Posted January 12, 2014 Share Posted January 12, 2014 I agree. Just waiting for RAID 5/6 to be stable in BTRFS. Same here. I think this will be the best compromise of features. I'm just afraid that it will take forever to be labeled "stable" and/or there won't be a nice webGUI to administer it. Hoping that Open Media Vault takes care of the latter. Quote Link to comment
drawz Posted January 12, 2014 Share Posted January 12, 2014 I agree. Just waiting for RAID 5/6 to be stable in BTRFS. Same here. I think this will be the best compromise of features. I'm just afraid that it will take forever to be labeled "stable" and/or there won't be a nice webGUI to administer it. Hoping that Open Media Vault takes care of the latter. Decided to see if there was any new info on BTRFS RAID 5/6 and came across the following post: http://www.phoronix.com/scan.php?page=news_item&px=MTU2NDQ and straight from the developers mouth: http://www.mail-archive.com/[email protected]/msg30103.html Quote Link to comment
grumpybutfun Posted January 12, 2014 Share Posted January 12, 2014 Decided to see if there was any new info on BTRFS RAID 5/6 and came across the following post: http://www.phoronix.com/scan.php?page=news_item&px=MTU2NDQ and straight from the developers mouth: http://www.mail-archive.com/[email protected]/msg30103.html Thanks for sharing... Great Info. BTRFS had a ton of updates / bug fixes / speed increases in Linux Kernel 3.12 Series. Glad the BTRFS RAID people are back at it. Quote Link to comment
Kryspy Posted January 12, 2014 Author Share Posted January 12, 2014 Now it has been a while since I did some reading up on BTRFS but I thought even in its current state you still got some pretty good customizable redunancy. Borrowing cues from the likes of Greyhole sort of you could choose your level of redunancy per volume or file. Kryspy Quote Link to comment
grumpybutfun Posted January 12, 2014 Share Posted January 12, 2014 Now it has been a while since I did some reading up on BTRFS but I thought even in its current state you still got some pretty good customizable redunancy. Borrowing cues from the likes of Greyhole sort of you could choose your level of redunancy per volume or file. Like ZFS... BTRFS is a File System and RAID built into one. BTRFS File System - Is Stable and has been for awhile. openSUSE is making it default in their next release and I believe Fedora is too. In the File System, you have checksums on data and metadata, scrub, snapshots, online filesystem defragmentation, batch, or out-of-band deduplication, etc. I was speaking directly to the RAID side of BTRFS. BTRFS is stable on raid0, raid1, raid10 just not in raid 5 or 6. Quote Link to comment
grumpybutfun Posted January 13, 2014 Share Posted January 13, 2014 What is the work involved in creating an overlay type of product with the unRAID MD system? Have you looked at SnapRAID? For Home Media Servers... It's hard to beat. Free, file checksum, encryption, fix silent errors, up to six drives for parity, each disk has a filesystem like unRAID, only spin up one drive, can use with any drive that already has data, actively developed, use any number of drives, sizes, add /remove drives no problem, it doesn't "take over" your Server / OS and you can install it in any Linux Distro. It's not a realtime RAID like unRAID or ZFS but if your server is used mostly for TV Shows and Movies... so what. Basically you back up your "raid" every night (once a week or whatever). All your new TV Shows / Movies / Music / etc. are backed up to your parity drive(s). Set it up to email you with the status report or whatever. Quote Link to comment
Kryspy Posted January 13, 2014 Author Share Posted January 13, 2014 Funny you should ask. That is exactly what I am using at the moment. Ubuntu 12.04 server, snapRAID with AUFS for pooling. Nightly syncs with a friendly e-mail to verify everything is okay. Kryspy Quote Link to comment
grumpybutfun Posted January 13, 2014 Share Posted January 13, 2014 Funny you should ask. That is exactly what I am using at the moment. Ubuntu 12.04 server, snapRAID with AUFS for pooling. Nightly syncs with a friendly e-mail to verify everything is okay. Nice setup. Why are you looking to change? Bored and want a new adventure? Quote Link to comment
Kryspy Posted January 13, 2014 Author Share Posted January 13, 2014 Why are you looking to change? Bored and want a new adventure? You got it. I've been using Ubuntu server for a while on an off. Lately been more on and have retired my unRAID setup. Kryspy Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.