February 2, 201115 yr Hey everyone, Figured I'd post this image... What you will see is a new storage system I implemented in my professional career. It's an Isilon (www.isilon.com) storage cluster with 48TB of RAW space (43TB after adjustment).. Best of all, it's shown to the end user as 1 big box o' storage. No partitions, no LUNs, no volumes, no aggregates, just pure, unadulterated contiguous storage. It ain't cheap, but the whole cluster took only 20 minutes (after rack installation/cabling) to put into production. In my 15 years of IT, this is by far the quickest deployment i've ever done. Expansion is a question of buying another node, plugging it in, clicking a button in the interface and voila! System scales to over 5PB if you want it and uses 40Gbps Infiniband as the inter-connect. Will also do 10Gbps Ethernet and iSCSI on the same "big box". Management couldn't be easier. Thought you'd guys might enjoy this. Cheers,
February 2, 201115 yr Nice! We were looking at this once our Promise units near retirement.... Although the Promise units are nice, they don't quite have the flexibility that these Isilon units have. Jealous
February 2, 201115 yr I have about 500 10 bath coins .. would i have enough ? or do i need to fill some more piggie's ?
February 2, 201115 yr hah cool what I find even more impressive, is that the users can be trusted with an open range of 43TB
February 2, 201115 yr Fun stuff. I just put in an EMC Data Domain (25tb raw) - it compresses data really well... our compression rate is about 83 percent!!! And I put in an entire rack of EMC Clarion/Celera/Centera storage... Forget a bunch of old school IBM mainframe disk!
February 2, 201115 yr our quantum dxi's are deduplicating and compressing to around 12:1 to 14:1. Just looking at one there, 977TB stored in 78TB -silly numbers Maybe unraid v10?
February 2, 201115 yr The nature of the types of data unRaid users store on their arrays (video and music) do not compress very well.
February 3, 201115 yr The nature of the types of data unRaid users store on their arrays (video and music) do not compress very well. Compress, no... dedupe... maybe. The big enterprise stuff dedupes at a block level instead of basic file level.... My Exchange databases (9.7tb) are stored on 337gb of disk.
February 3, 201115 yr The nature of the types of data unRaid users store on their arrays (video and music) do not compress very well. Compress, no... dedupe... maybe. The big enterprise stuff dedupes at a block level instead of basic file level.... My Exchange databases (9.7tb) are stored on 337gb of disk. When backing up enterprise PCs, it is easy to de-dupe. Nearly all will be the same OS, with the same files. The only "duplicate" files I have are some pictures where I just dumped the camera's flash card to a dated folder on the disk and did not bother deleting the pictures already on it before taking more. Now, there are a few movies with "flashbacks" and repeating segments (We watched "Groundhog Day" this evening) but I doubt they would compress or de-dupe at all. Joe L.
February 3, 201115 yr The nature of the types of data unRaid users store on their arrays (video and music) do not compress very well. Compress, no... dedupe... maybe. The big enterprise stuff dedupes at a block level instead of basic file level.... My Exchange databases (9.7tb) are stored on 337gb of disk. When backing up enterprise PCs, it is easy to de-dupe. Nearly all will be the same OS, with the same files. The only "duplicate" files I have are some pictures where I just dumped the camera's flash card to a dated folder on the disk and did not bother deleting the pictures already on it before taking more. Now, there are a few movies with "flashbacks" and repeating segments (We watched "Groundhog Day" this evening) but I doubt they would compress or de-dupe at all. Joe L. I'm not so sure about that. Block level dedupe is magical. I'm going to bring a few 8 or 9gb .mkv movies in and stick them on there and see what happens
February 3, 201115 yr I'm not so sure about that. Block level dedupe is magical. I'm going to bring a few 8 or 9gb .mkv movies in and stick them on there and see what happens Skeptical but open minded. Looking forward to your results. What is the "block size"?
February 3, 201115 yr I'm not so sure about that. Block level dedupe is magical. I'm going to bring a few 8 or 9gb .mkv movies in and stick them on there and see what happens Skeptical but open minded. Looking forward to your results. What is the "block size"? Well.. with a block size small enough, there will be lots of duplicates... Try a block size of "1" byte and I'll bet you find lots of duplicates in many files. You should be able to store only one copy of the potential 256 unique "bytes" and then reference them for all the others. Theoretically it will work for the de-duplication, but overhead and efficiency might be poor. Joe L.
February 4, 201115 yr Author Deduping with 1 byte blocks would create so much overhead.. it's almost imaginable
February 4, 201115 yr Deduping with 1 byte blocks would create so much overhead.. it's almost imaginable I know... but think of how many dupes you'd eliminate. It is not as bad as you might think since you only need 1 byte to point to the value to be used instead of the duped block, and 8 bytes to address the "block" being replaced. That's a 9-to-1 ratio...
February 14, 201115 yr If the whole thing can show up as one drive, can I put 20 of them into unRAID? I like the cut of your jib.
Archived
This topic is now archived and is closed to further replies.