Stokkes Posted February 2, 2011 Share Posted February 2, 2011 Hey everyone, Figured I'd post this image... What you will see is a new storage system I implemented in my professional career. It's an Isilon (www.isilon.com) storage cluster with 48TB of RAW space (43TB after adjustment).. Best of all, it's shown to the end user as 1 big box o' storage. No partitions, no LUNs, no volumes, no aggregates, just pure, unadulterated contiguous storage. It ain't cheap, but the whole cluster took only 20 minutes (after rack installation/cabling) to put into production. In my 15 years of IT, this is by far the quickest deployment i've ever done. Expansion is a question of buying another node, plugging it in, clicking a button in the interface and voila! System scales to over 5PB if you want it and uses 40Gbps Infiniband as the inter-connect. Will also do 10Gbps Ethernet and iSCSI on the same "big box". Management couldn't be easier. Thought you'd guys might enjoy this. Cheers, Quote Link to comment
speeding_ant Posted February 2, 2011 Share Posted February 2, 2011 Nice! We were looking at this once our Promise units near retirement.... Although the Promise units are nice, they don't quite have the flexibility that these Isilon units have. Jealous Quote Link to comment
kizer Posted February 2, 2011 Share Posted February 2, 2011 Here I was hoping for a big old box with wires everywhere. Quote Link to comment
sacretagent Posted February 2, 2011 Share Posted February 2, 2011 I have about 500 10 bath coins .. would i have enough ? or do i need to fill some more piggie's ? Quote Link to comment
Carpet3 Posted February 2, 2011 Share Posted February 2, 2011 That's impressive Quote Link to comment
kal Posted February 2, 2011 Share Posted February 2, 2011 hah cool what I find even more impressive, is that the users can be trusted with an open range of 43TB Quote Link to comment
LVLAaron Posted February 2, 2011 Share Posted February 2, 2011 Fun stuff. I just put in an EMC Data Domain (25tb raw) - it compresses data really well... our compression rate is about 83 percent!!! And I put in an entire rack of EMC Clarion/Celera/Centera storage... Forget a bunch of old school IBM mainframe disk! Quote Link to comment
kal Posted February 2, 2011 Share Posted February 2, 2011 our quantum dxi's are deduplicating and compressing to around 12:1 to 14:1. Just looking at one there, 977TB stored in 78TB -silly numbers Maybe unraid v10? Quote Link to comment
SSD Posted February 2, 2011 Share Posted February 2, 2011 The nature of the types of data unRaid users store on their arrays (video and music) do not compress very well. Quote Link to comment
LVLAaron Posted February 3, 2011 Share Posted February 3, 2011 The nature of the types of data unRaid users store on their arrays (video and music) do not compress very well. Compress, no... dedupe... maybe. The big enterprise stuff dedupes at a block level instead of basic file level.... My Exchange databases (9.7tb) are stored on 337gb of disk. Quote Link to comment
Joe L. Posted February 3, 2011 Share Posted February 3, 2011 The nature of the types of data unRaid users store on their arrays (video and music) do not compress very well. Compress, no... dedupe... maybe. The big enterprise stuff dedupes at a block level instead of basic file level.... My Exchange databases (9.7tb) are stored on 337gb of disk. When backing up enterprise PCs, it is easy to de-dupe. Nearly all will be the same OS, with the same files. The only "duplicate" files I have are some pictures where I just dumped the camera's flash card to a dated folder on the disk and did not bother deleting the pictures already on it before taking more. Now, there are a few movies with "flashbacks" and repeating segments (We watched "Groundhog Day" this evening) but I doubt they would compress or de-dupe at all. Joe L. Quote Link to comment
LVLAaron Posted February 3, 2011 Share Posted February 3, 2011 The nature of the types of data unRaid users store on their arrays (video and music) do not compress very well. Compress, no... dedupe... maybe. The big enterprise stuff dedupes at a block level instead of basic file level.... My Exchange databases (9.7tb) are stored on 337gb of disk. When backing up enterprise PCs, it is easy to de-dupe. Nearly all will be the same OS, with the same files. The only "duplicate" files I have are some pictures where I just dumped the camera's flash card to a dated folder on the disk and did not bother deleting the pictures already on it before taking more. Now, there are a few movies with "flashbacks" and repeating segments (We watched "Groundhog Day" this evening) but I doubt they would compress or de-dupe at all. Joe L. I'm not so sure about that. Block level dedupe is magical. I'm going to bring a few 8 or 9gb .mkv movies in and stick them on there and see what happens Quote Link to comment
SSD Posted February 3, 2011 Share Posted February 3, 2011 I'm not so sure about that. Block level dedupe is magical. I'm going to bring a few 8 or 9gb .mkv movies in and stick them on there and see what happens Skeptical but open minded. Looking forward to your results. What is the "block size"? Quote Link to comment
Joe L. Posted February 3, 2011 Share Posted February 3, 2011 I'm not so sure about that. Block level dedupe is magical. I'm going to bring a few 8 or 9gb .mkv movies in and stick them on there and see what happens Skeptical but open minded. Looking forward to your results. What is the "block size"? Well.. with a block size small enough, there will be lots of duplicates... Try a block size of "1" byte and I'll bet you find lots of duplicates in many files. You should be able to store only one copy of the potential 256 unique "bytes" and then reference them for all the others. Theoretically it will work for the de-duplication, but overhead and efficiency might be poor. Joe L. Quote Link to comment
Stokkes Posted February 4, 2011 Author Share Posted February 4, 2011 Deduping with 1 byte blocks would create so much overhead.. it's almost imaginable Quote Link to comment
Joe L. Posted February 4, 2011 Share Posted February 4, 2011 Deduping with 1 byte blocks would create so much overhead.. it's almost imaginable I know... but think of how many dupes you'd eliminate. It is not as bad as you might think since you only need 1 byte to point to the value to be used instead of the duped block, and 8 bytes to address the "block" being replaced. That's a 9-to-1 ratio... Quote Link to comment
fitbrit Posted February 5, 2011 Share Posted February 5, 2011 If the whole thing can show up as one drive, can I put 20 of them into unRAID? Quote Link to comment
Rajahal Posted February 14, 2011 Share Posted February 14, 2011 If the whole thing can show up as one drive, can I put 20 of them into unRAID? I like the cut of your jib. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.